Synthesis of reinforcement learning, neural networks and PI control applied to a simulated heating coil

An accurate simulation of a heating coil is used to compare the performance of a proportional plus integral (PI) controller, a neural network trained to predict the steady-state output of the PI controller, a neural network trained to minimize the n-step ahead error between the coil output and the set point, and a reinforcement learning agent trained to minimize the sum of the squared error over time. Although the PI controller works very well for this task, the neural networks produce improved performance. The reinforcement learning agent, when combined with a PI controller, learned to augment the PI control output for a small number of states for which control can be improved.